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Abstract 

Naturalistic  decision  making  (NDM)  has  become  established  as  a  methodological  and 
theoretical  perspective.  It  describes  how  practitioners  actually  make  decisions  in 
complex  domains.  However,  NDM  theories  tend  to  focus  on  the  human  agents  in  the 
system.  We  extend  the  NDM  perspective  to  include  the  technological  agents  in 
complex  systems  and  introduce  the  dynamic  model  of  situated  cognition.  We  describe 
the  general  characteristics  of  NDM  and  the  field  of  situated  cognition,  and  provide  a 
detailed  description  of  our  model.  We  then  apply  the  model  to  a  recent  accident  in 
which  a  US  Navy  submarine  (USS  Greeneville )  collided  with  a  Japanese  fishing 
vessel  ( Ehime  Maru).  The  discussion  of  the  accident  illustrates  how  decisions  made 
are  often  a  result  of  the  interaction  between  a  variety  of  technological  and  human 
agents  and  how  errors  introduced  into  the  complex  system  can  propagate  through  it 
in  unintended  ways.  We  argue  that  the  dynamic  model  of  situated  cognition  can  be 
used  to  describe  activities  in  virtually  any  complex  domain. 
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Since  the  first  naturalistic  decision-making  conference  in  Dayton,  OH,  in 
September  1989  and  the  publication  of  Decision  making  in  action:  Methods 
and  models  in  1993,  naturalistic  decision  making  (NDM)  has  become  estab¬ 
lished  as  a  methodological  and  theoretical  perspective.  Because  it  describes 
how  practitioners  actually  make  decisions  in  complex  domains  rather  than 
how  they  ought  to  make  decisions,  many  fields  of  practice  have  adopted  NDM 
as  a  doctrinal  framework.  For  example,  the  US  Army  has  incorporated  aspects 
of  NDM  in  its  latest  version  of  Field  manual  (FM)  6-0,  Mission  command: 
Command  and  control  of  army  forces  (2003).  While  NDM  represents  a  major 
step  forward  in  our  understanding  of  the  activities  in  various  fields  of  practice, 
the  focus  of  NDM  has  been  on  the  human  agents  in  complex  systems  and  has 
not  emphasized  the  influence,  contributions,  and  modeling  of  the  techno¬ 
logical  aspects  of  these  systems. 

The  article  begins  with  a  discussion  of  the  characteristics  inherent  in 
environments  in  which  naturalistic  decision  making  is  typically  employed.  A 
model  of  NDM  is  then  described  and  contrasted  with  what  the  authors  refer 
to  as  the  dynamic  model  of  situated  cognition  (DMSC).  The  DMSC  captures 
both  the  human  and  technological  components  of  complex  systems  into  a 
single  model  and  illustrates  how  the  decision  making  of  a  human  is  influenced 
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by  both  technological  agents  and  other  human  agents.  We  use  an  actual 
military  accident,  the  collision  of  the  USS  Greeneville  with  the  Ehime  Mam, 
to  trace  the  flow  of  information  through  the  model,  to  highlight  what  decisions 
were  made,  how  the  decisions  were  made,  and  how  the  technological  and 
human  aspects  of  the  system  contributed  to  the  accident.  Thus,  the  model 
couples  NDM  theory  with  a  conceptual  model  that  includes  technology  to 
provide  a  more  robust  insight  into  total  system  performance. 


Characteristics  of  Naturalistic  Decision  Making 

Orasanu  and  Connolly  (1993)  describe  eight  conditions  in  which  naturalistic 

decision  making  is  typically  employed. 

•  Problems  tend  to  be  ill-structured.  That  is,  for  some  real-world  problems, 
it  is  not  easy  or  even  possible  to  identify  causes  and  potential  courses  of 
actions. 

•  The  conditions  are  uncertain  and  dynamic.  The  situation  is  continually 
changing,  making  it  difficult  to  assess  what  is  happening.  Static  repre¬ 
sentations  of  the  system  are  of  little  use  since  the  situation  is  changing 
so  quickly. 

•  The  multiple  goals  may  be  ill-defined,  may  be  in  conflict,  or  may  shift 
over  time.  A  military  commander  might  have  multiple  goals:  defeating 
the  enemy,  minimizing  collateral  damage,  causing  no  harm  to  non- 
combatants,  and  protecting  soldiers  from  harm.  Not  only  may  these  goals 
change  from  time  to  time,  they  may,  in  fact,  conflict  with  one  another. 

•  NDM  acknowledges  the  existence  of  action  and  feedback  loops.  Decisions 
are  not  discrete  events  but  happen  amidst  the  flow  of  activity  in  a  system 
and  are  impacted  by  the  decisions  and  activity  that  precede  them. 

•  Decision  makers  must  respond  in  real  time  to  changes  in  the  system. 
Diagnosis  of  problems  and  system  control  often  happen  simultaneously. 
Even  while  a  pilot  is  trying  to  assess  an  anomaly  or  malfunction  he  or 
she  must  still  fly  the  aircraft. 

•  The  domains  in  which  NDM  processes  are  employed  often  involve  high 
stakes.  Examples  of  these  domains  include  firefighting,  military  command 
and  control,  air  traffic  control,  hospital  operating  rooms,  nuclear  power 
plants,  and  weather  forecasting. 

•  Multiple  players  interact  in  the  decision-making  process.  These  players 
may  have  either  shared  or  different  views  of  the  situation.  They  must 
cooperate  with  one  another  and  update  each  other  in  order  to  perform 
optimally.  While  many  researchers  view  these  multiple  players  as  humans, 
the  authors  of  this  article  agree  with  other  researchers  who  believe  that 
these  players  must  include  both  machine  and  human  agents.  Interactions 
between  humans  and  machines  are  rife  in  complex  systems.  These 
interactions  can  lead  to  situation  assessments  that  result  in  decisions  by 
either  the  human  or  the  machine.  Unfortunately,  either  the  machine  or  the 
human  may  reach  an  incorrect  decision  based  on  the  information  they 
receive  from  another  (human  or  machine)  agent. 
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•  NDM  typically  examines  activities  that  are  embedded  in  organizations. 
Organizations  have  their  own  unique  cultures  which  manifest  themselves 
in  accepted  norms,  policies,  guidelines,  directives,  standard  operating 
procedures,  and  doctrine.  Various  aspects  of  these  cultures  may  be 
communicated  explicitly  (verbally  or  in  written  documents)  or  implicitly 
(through  behavioral  modeling  or  system  design). 

In  general,  decision-making  research  has  progressed  from  normative  to 
descriptive  to  naturalistic.  Researchers  observed  that  people  often  did  not 
make  optimal  decisions  so  they  began  to  investigate  how  and  why  deviations 
occurred.  While  these  studies  identified  several  heuristics  and  biases  used  by 
decision  makers,  their  external  validity  was  doubtful  because  they  often  used 
naive  participants  to  perform  contrived,  discrete  tasks.  As  the  eight  charac¬ 
teristics  above  suggest,  NDM  theories  generally  describe  how  decisions  are 
made  in  operational  settings.  During  the  last  two  decades,  NDM  research  has 
been  conducted  in  numerous  fields  of  practice,  generating  a  broad  empirical 
base  and  a  variety  of  theories.  Perhaps  the  best-known  naturalistic  decision¬ 
making  theory  is  Klein’s  recognition-primed  decision  model. 


A  Model  of  Naturalistic  Decision  Making 

The  recognition-primed  decision  (RPD)  model  was  developed  by  Klein  and 
his  colleagues  (Klein  et  al.  1986)  based  on  their  observations  of  decision 
makers  in  operational  settings.  The  model  describes  how  experts  use  their 
experiences  to  arrive  at  decisions  quickly  and  without  the  computational  (i.e. 
cost-benefit  or  utility)  analysis  of  traditional  normative  decision-making 
approaches  (Raiffa  1968).  RPD  employs  situation  assessment  to  generate  a 
likely  course  of  action  and  then  uses  mental  simulation  to  envision  and 
evaluate  the  course  of  action.  Klein  (1993a)  described  three  versions  of  RPD, 
which  differ  in  their  complexity.  In  the  simplest  case,  the  situation  confronting 
the  decision  maker  is  very  similar  to  a  situation  or  a  class  of  situations 
previously  experienced.  The  decision  maker  invokes  the  solution  from  the 
previous  situation  in  the  present  circumstance.  In  the  more  complex  situation, 
the  decision  maker  uses  mental  simulation  to  envision  how  the  solution  might 
work  and  what  problems  might  be  encountered.  If  the  decision  maker 
envisions  a  problem,  the  solution  is  modified  and  the  mental  simulation  is 
repeated  with  the  revised  solution.  In  the  most  complex  situation,  an  evalua¬ 
tion  process  determines  the  solution  to  be  inadequate  and  the  next  most 
appropriate  solution  must  be  identified  and  evaluated.  This  serial  process 
continues  until  an  adequate  solution  is  found  and  implemented.  According  to 
Klein,  a  key  aspect  of  all  three  cases  is  the  process  by  which  the  decision 
maker  recognizes  the  situation. 

As  a  general  strategy,  RPD  appears  to  be  quite  robust,  having  been  studied 
in  a  wide  variety  of  operational  settings.  Many  researchers  have  sought  to 
leverage  and  extend  the  work  already  completed  on  RPD.  Klein  (1993b) 
himself  proposed  consolidating  five  NDM  models  into  one  synthesized 
model.  Some  have  proposed  alternative  NDM  models  (Lipshitz  1993). 
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Leedom  (2003)  considered  the  challenges  that  arise  when  RPD  is  extended 
to  collective  decision-making  activities  in  complex  organizations.  Others  have 
sought  to  expand  on  the  processes  embedded  in  the  model.  Noble  (1993),  for 
example,  further  described  the  process  of  situation  assessment.  McLennan 
and  Omodei  (1996)  proposed  a  prepriming  process  which  precedes  situation 
assessment  and  frames  that  assessment  within  a  particular  context.  Still  others 
have  attempted  to  build  computational  models  of  RPD  (Warwick  et  al.  2002). 

We  acknowledge  the  outstanding  efforts  of  all  these  researchers  and  believe 
there  is  great  merit  in  NDM,  in  general,  and  RPD,  in  particular.  We  propose 
to  extend  the  NDM  framework  into  an  area  which  we  believe  has  been 
somewhat  neglected.  In  general,  NDM  models  do  not  discuss  the  technolog¬ 
ical  aspects  of  complex  systems  and  the  influence  they  have  on  decision 
making.  We  firmly  believe  that  the  technological  aspects  of  a  complex  system 
should  be  included  in  models  of  naturalistic  decision  making.  Much  of  what 
is  decided  is  based  on  what  is  perceived  by  the  decision  maker.  And  much  of 
what  is  perceived  is  based  on  the  design  of  the  technology  in  the  system.  For 
example,  technology  can  aid  or  impede  decision  makers  by  representing  the 
environment  more  or  less  accurately.  Decision  makers  may  be  passive 
recipients  of  data  presented  by  the  technology  or  they  may  be  in  a  position 
to  shape  and  direct  the  technology.  In  either  case,  there  is  an  inextricable  link 
between  the  technological  components  and  the  human  agents  in  a  complex 
system.  We  now  describe  a  conceptual  model  of  situated  cognition  that 
incorporates  both  technological  and  human  aspects  of  a  complex  system. 


A  Dynamic  Model  of  Situated  Cognition 

Traditional  models  of  cognitive  processing  and  decision  making  often  include 
modules  which  represent  sensory  input,  different  types  of  memory,  encoding, 
response  selection  and  execution,  and  attention  (Wickens  et  al.  2004). 
Proponents  of  such  models  often  use  controlled  laboratory  tests  to  study 
individuals  engaged  in  activities  that  focus  on  selected  components  of  these 
models.  These  studies  often  use  college  students  as  participants  and  involve 
performing  simple  tasks  for  which  no  experience  is  necessary.  While  it  is 
essential  that  such  work  be  conducted,  this  type  of  research  neglects  the  types 
of  cognitive  activities  that  experienced  people  use  in  contexts  with  which  they 
are  familiar.  Cacciabue  and  Hollnagel  (1995)  refer  to  this  level  of  analysis  as 
‘macrocognition’.  Klein  et  al.  (2003)  include  the  following  as  macrocognitive 
activities:  decision  making,  uncertainty  management,  mental  simulation, 
sensemaking,  situation  awareness,  attention  management,  problem  detection, 
planning,  and  option  generation. 

A  related  field  of  research,  which  also  values  the  importance  of  context  and 
expertise,  is  referred  to  as  situated  cognition.  Learning  theorists  combined 
knowledge  from  cognitive  psychology,  sociology,  critical  psychology,  anthro¬ 
pology,  and  sociocultural  theory  to  investigate  the  ‘fundamental  processes  of 
cognition  as  social  and  situated  activity’  (Kirshner  and  Whitson  1997:  3). 
Lemke  (1997:  38)  describes  recent  research  and  thought  in  situated  cognition 
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as  the  interplay  of  multiple  individuals  and  their  environment.  People  engaged 
in  normal  activities: 

‘are  functioning  in  microecologies,  material  environments  endowed  with  cultural 
meanings;  acting  and  being  acted  on  directly  or  with  the  mediation  of  physical-cultural 
tools  and  cultural-material  systems  of  words,  signs,  and  other  symbolic  values.  In 
these  activities,  “things”  contribute  to  solutions  every  bit  as  much  as  “minds”  do; 
information  and  meaning  [is]  coded  into  configurations  of  objects,  material 
constraints,  and  possible  environmental  options,  as  well  as  verbal  routines  and 
formulas  or  “mental”  operations.’ 

Lemke  (1997:  38)  further  states  that 

‘our  activity,  our  participation,  our  “cognition”  is  always  bound  up  with,  codependent 
with,  the  participation  and  the  activity  of  [0]thers,  be  they  persons,  tools,  symbols, 
processes,  or  things.  How  we  participate,  what  practices  we  come  to  engage  in,  is  a 
function  of  the  whole  community  ecology,  or  at  least  of  those  parts  of  it  we  join  in 
with.’ 

Learning  theorists  are  not  the  only  people  who  have  expressed  interest  in 
situated  cognition.  AI  (artificial  intelligence)  researchers,  attempting  to  model 
human  cognition  and  build  ‘intelligent’  machines,  have  seized  upon  the  term. 
They  describe  situated  cognition  as 

‘the  study  of  how  human  knowledge  develops  as  a  means  of  coordinating  activity 
within  activity  itself.  This  means  that  feedback  —  occurring  internally  and  with  the 
environment  over  time  —  is  of  paramount  importance.  Knowledge  therefore  has  a 
dynamic  aspect  in  both  formation  and  content.’  (Clancey  1997:  4) 

The  AI  community  emphasizes  the  importance  of  feedback  and  the  dynamic 
nature  of  the  way  in  which  knowledge  is  formed  and  utilized. 

‘In  summary,  the  term  situated  cognition  emphasizes  that  perceptual-motor  feedback 
mechanisms  causally  relate  animal  cognition  to  the  environment  and  action  in  a  way 
that  a  mechanism  based  on  logical  (descriptive)  inference  alone  does  not  capture. 
Embodiment  is  more  than  receiving  signals  from  the  environment  or  modifying  the 
environment  (such  as  a  process  control  program  in  a  manufacturing  plant).  Being 
situated  involves  a  causal,  in-the-moment  coupling  within  internal  organizing 
(forming  new  coordinations)  and  between  internal  and  external  organizing  (changing 
stuff  in  the  world).  Hence,  new  ways  of  seeing  and  ways  of  making  changes  to  the 
world  develop  together.  Time  sensitivity  (reactiveness)  doesn’t  mean  just  reasoning 
about  events  —  reasoning  exists  as  a  physical  activity  in  time  (even  in  the 
imagination),  and  past  reasoning  experiences  are  causally  influencing  reasoning  as 
it  unfolds  in  time.’  (Clancey  1997:  344) 

Learning  theorists  and  AI  researchers  alike  understand  the  role  of  context 
in  knowledge  acquisition.  Similarly,  their  descriptions  of  situated  cognition 
also  include  a  dynamic  environment  in  which  multiple  individuals  interact 
with  one  another  and  with  tools,  feedback  mechanisms,  and  the  use  of 
previous  knowledge  to  direct  behavior  and  guide  the  formation  of  new 
knowledge.  The  dynamic  model  of  situated  cognition  integrates  human  and 
machine  components  and  is  intended  to  describe  what  happens  in  operational 
environments  with  experienced  people  who  are  engaged  in  goal-directed 
behavior.  Thus,  their  actions  and  their  cognitive  processes  are  situated  in  a 
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Figure  1 .  Dynamic  Model  of  Situated  Cognition 


particular  context.  It  is  assumed  that  this  context  is  continually  evolving  (or 
dynamic)  and,  therefore,  is  continually  influencing  the  cognitive  activities  of 
the  humans  in  the  system.  In  operational  environments,  it  is  often  the 
technological  portion  of  the  system  that  triggers  the  activity  of  the  human 
portion  of  the  system. 

The  dynamic  model  of  situated  cognition  (Figure  1)  emerged  as  an  attempt 
to  illustrate  the  relationship  between  technological  systems  and  human 
perceptual  and  cognitive  processes,  which  has  received  only  marginal 
attention  in  the  NDM  and  the  situated  cognition  literature.  The  largest  oval 
on  the  left  side  of  the  figure  (Oval  1)  depicts  everything  that  exists  in  the 
environment  and  may  also  be  referred  to  as  ground  truth  or  as  a  God’s  eye 
view  of  reality  at  that  precise  moment  in  time.  Oval  1  is  a  completely  accurate 
picture  that  is  continually  being  updated  and  is,  therefore,  dynamic.  In  a 
military  C2  scenario,  the  various  shapes  represent  individual  data  elements 
for  friendly  and  enemy  entities,  friendly  and  enemy  intent,  and  data  on 
noncombatants,  weather,  and  terrain. 

The  next  oval  (Oval  2)  illustrates  those  data  elements  detected  by  tech¬ 
nological  systems.  Note  that  this  oval  is,  in  the  best  of  cases,  a  subset  of  the  first 
oval.  Data  elements  that  are  the  same  in  Ovals  1  and  2  represent  the  accurate 
detection  of  items.  We  refer  to  this  as  technological  accuracy.  Not  everything 
in  Oval  1  is  detected  accurately  for  a  variety  of  reasons  (see  Figure  2).  Sensor 
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Figure  2. 

Not  all  data  in 
Oval  1  Dynamic 
Model  of  Situated 
Cognition  flow 
accurately  to 
Ovals  2  &  3.  Data 
can  be  missed  or 
misidentified  by 
technological 
systems  on  the  left 
side  of  the  model. 


Accurate  Inaccurate 


Flow  of  data  from 
Oval  1  to  Ovals  2  &  3 


Missed 


\ 


Misidentified 


Sensors: 

•  Insufficient  in  number 

•  Malfunction 

•  Lack  of  sensitivity 

•  Lack  of  specificity 


Flawed  algorithm 
Spoofing 


can  either  miss  or  misidentify  data  in  Oval  1 .  Data  may  be  missed  for  the 
following  reasons: 

•  There  may  be  an  insufficient  number  of  sensors  to  cover  the  environment. 

•  The  technology  may  not  be  sensitive  enough  to  detect  certain  classes  of 
data. 

•  The  technology  may  not  have  the  specificity  required  to  identify  certain 
classes  of  data. 

•  Sensors  may  malfunction. 

In  Figure  1 ,  Oval  2  includes  stars  and  circles  which  do  not  appear  in  Oval 
1 .  This  difference  represents  the  misidentification  that  can  occur  as  data  flow 
from  Oval  1  to  Oval  2.  Data  are  misidentified  either  because  the  underlying 
sensor  algorithm  is  flawed  or  because,  as  is  the  case  in  military  environments, 
the  enemy  may  purposely  misrepresent  their  location  and  resources  in  order 
to  deceive  friendly  commanders.  Because  the  data  elements  are  either  missed 
or  somehow  misidentified  as  the  data  flow  from  Oval  1  to  Oval  2,  the  resultant 
inaccuracy  may  be  propagated  throughout  the  rest  of  the  model.  And,  it  may 
not  be  apparent  to  the  decision  maker  what  portions  of  the  environment  are 
being  detected,  what  items  remain  undetected,  and  what  has  been  erroneously 
identified.  Issues  of  accuracy  and  certainty  permeate  throughout  the  model 
and  these  will  be  discussed  in  greater  detail  later  in  the  paper. 

Oval  3  depicts  the  data  displayed  on  the  local  decision  maker’s  workstation. 
These  data,  again,  are  potentially  an  inaccurate  and  uncertain  subset  of  what 
is  detected  by  the  available  technological  systems.  The  inaccuracies  and 
uncertainties  introduced  in  Oval  2  (i.e.  the  stars  and  circles)  may  propagate 
to  Oval  3.  In  addition,  the  technology  embedded  in  the  local  C2  system  may 
misrepresent,  through  erroneous  fusion  algorithms,  flawed  filtering  schemes, 
or  poorly  designed  displays,  what  is  present  (i.e.  the  cross  in  Oval  3). 

In  current  and  planned  C2  systems,  decision  makers  are  able  to  tailor  their 
displays  to  suit  their  individual  preferences.  The  capability  to  tailor  what  is 
displayed  is  one  way  designers  may  attempt  to  assist  decision  makers  in 
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coping  with  data  overload.  The  drawback  in  this  approach  is  that  it  is  not 
always  obvious  to  decision  makers  which  data  have  been  excluded  by  their 
preferences.  In  addition,  decision  makers  may  have  different  preferences 
selected,  resulting  in  divergent  and  possibly  confusing  views  of  the  environ¬ 
ment.  For  example,  at  a  recent  military  experiment,  the  automated  C2  system 
generated  over  9000  alert  messages  in  a  90-minute  period.  However,  the  four 
participants  in  the  C2  vehicle  had  tailored  their  workstations  independent  of 
one  another.  As  a  result,  only  a  few  hundred  of  the  alerts  were  actually 
displayed  on  the  workstations;  many  alerts  appeared  on  multiple  workstations 
while  literally  thousands  of  alerts  were  missed  altogether,  in  some  cases 
depriving  the  participants  of  vital  information. 

The  three  ovals  discussed  thus  far  represent  the  technological  portion  of 
the  model.  Data  displayed  on  the  local  C2  workstation  are  not  just  a  function 
of  how  the  decision  maker  has  configured  the  screen;  the  configuration  is 
also  based  on  how  the  system  designers  have  constructed  the  tools  employed 
by  the  decision  maker  to  cope  with  data  overload.  What  is  displayed  on 
the  screen  may  or  may  not  represent  the  most  important  data  detected  by  the 
sensor  array.  And  since  the  decision  maker’s  knowledge  of  the  environment 
is  directly  related  to  what  is  displayed  on  the  workstation,  that  knowledge 
may  or  may  not  be  an  accurate  reflection  of  the  actual  activity  in  the  environ¬ 
ment.  Further,  the  sensor  array  is  only  a  sample  or  a  subset  of  all  the  data  in 
the  environment.  That  which  is  sensed  is  incomplete  and  the  awareness 
constructed  from  it  will  be  incomplete.  Incomplete  does  not  necessarily  mean 
inaccurate.  If  the  right  data  are  sampled,  the  awareness  constructed  will  not 
be  complete  but  could  still  be  accurate.  However,  since  decision  makers 
usually  do  not  know  where  the  ‘holes’  are  in  the  friendly  and  enemy  sensor 
coverage,  they  are  ignorant  of  how  their  mental  model  differs  from  the  actual 
activities  and  events  of  the  environment. 

Three  dynamic  lenses  are  depicted  in  Figure  1.  These  lenses,  and  the  role 
they  play  in  the  dynamic  model  of  situated  cognition,  are  somewhat  analogous 
to  the  lens  model  developed  by  Bruns wik  (1947)  and  described  by  Beach 
(1997).  In  Bruns  wik’ s  model,  a  decision  maker  must  make  inferences  (Y ) 
about  events  (Y  )  by  using  available  cues  (C).  The  left  side  of  the  lens  includes 
descriptions  of  the  events  that  are  occurring  in  the  environment  and  the  cues 
that  represent  those  events.  The  right  side  of  the  Brunswik’s  lens  describes 
how  the  decision  maker  uses  the  cues  to  make  inferences  about  what  is 
happening  on  the  left  side  of  the  lens.  Hammond  modified  Brunswik’s  model 
to  include  multiple  lenses  (Beach  1997).  Some  decisions  might  have  multiple 
intermediate  decisions  that  must  be  reached.  In  this  Brunswikian  lens 
approach,  each  of  these  intermediate  decisions  is  represented  by  its  own  lens, 
the  output  of  which  feeds  forward  to  the  next  decision. 

The  lenses  in  the  dynamic  model  of  situated  cognition  combine  elements 
from  the  left  side  of  each  lens  with  informational  elements  ‘resident’  in  the 
decision  maker  and  feed  forward  information  or  decisions  to  the  next  portion 
of  the  model.  Although  the  informational  elements  are  the  same  in  each  lens, 
the  placement  of  the  lenses  in  the  model  suggests  that  different  functions  are 
performed  by  each  lens.  As  is  the  case  with  the  human  visual  lens,  perceptual 
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distortions  may  result  from  asymmetries  and  flaws  in  the  refining  process. 
Lens  A,  the  lens  between  Ovals  3  and  4,  directs  attention  to  selected  incoming 
stimuli.  These  stimuli  are,  in  most  cases,  either  visual  or  auditory.  Between 
Ovals  4  and  5  is  Lens  B,  which  influences  how  data  are  organized  into 
information.  The  lens  between  Ovals  5  and  6,  Lens  C,  guides  the  process  of 
extrapolating  current  information  into  predictions  about  the  future. 

There  are  at  least  six  classes  of  information  embedded  in  the  lenses  that 
influence  how  decision  makers  perceive,  comprehend,  and  make  predictions 
about  activities  on  the  battlefield.  Individual  states  and  traits  represent  those 
relatively  enduring  (e.g.  intelligence  or  personality)  and  transient  (e.g.  fatigue) 
characteristics  of  an  individual  that  affect  decision  making.  For  a  more 
detailed  discussion  of  human  trait  and  state  measurements,  see  Miller  et  al. 
(2003).  Social  factors  include  issues  ranging  from  small  group  dynamics  to 
cultural  differences  that  might  exist  among  decision  makers.  The  local  context 
influences  the  data  to  which  a  decision  maker  will  attend.  The  plan  represents 
the  specific  goals  to  be  achieved  and  the  means  by  which  they  will  be 
achieved.  Guidelines  represent  general  procedures  to  which  decision  makers 
may  refer  if  the  plan  is  underspecified.  Experience  refers  to  previous  activities 
in  which  a  decision  maker  has  engaged.  Decision  makers  rely  (either 
consciously  or  unconsciously)  on  prior  experiences  to  influence  how  they 
direct  their  attention.  In  particular,  data  which  were  determined  to  be  useful 
in  previous  situations  may  also  prove  useful  in  the  current  situation.  Hence, 
these  data  may  be  valued  over  other  data  available  to  the  decision  maker. 
Together,  these  six  classes  of  information  influence  what  is  perceived  by  the 
decision  maker  (Oval  4). 

The  oval  to  the  immediate  right  of  the  first  lens  (Oval  4)  represents  all  the 
data  actually  perceived  by  the  decision  maker.  The  small  gray  dashed  ovals 
embedded  in  Ovals  1,  2,  and  3  portray  the  idea  that  data  perceived  in  Oval  4 
are  only  a  portion  of  data  available  in  previous  ovals.  Perceived  data  are  a 
small  subset  of  the  data  available  in  the  environment,  the  sensor  array,  the 
configuration  of  the  local  C2  display,  and  are  based  on  the  characteristics  of 
the  lens.  The  data  perceived  may  be  a  result  of  active  or  passive  processes. 
Active  processes  refer  to  data  requested  or  ‘pulled’  by  the  decision  maker 
and,  therefore,  a  result  of  goal-directed  behavior.  Passive  processes  refer  to 
data  provided  or  ‘pushed’  to  the  decision  maker  without  a  request  and, 
therefore,  may  or  may  not  be  relevant  to  goal  accomplishment. 

Perceived  data  are  of  little  value  to  the  decision  maker  until  they  are 
processed  further  (i.e.  comprehended).  The  term  comprehension  is  used  generi- 
cally  in  this  paper  to  refer  to  cognitive  processes  such  as  fusion,  integration, 
analysis,  explanation,  interpretation,  and  pattern  recognition  (Endsley  1995). 
Comprehension  is  illustrated  in  Oval  5.  The  same  lens  components  (i.e. 
individual  states  and  traits,  social  factors,  local  context,  plans,  guidelines,  and 
experience)  that  directed  attention  and  led  to  perception  may  also  influence 
comprehension.  In  Oval  5  of  the  figure,  the  friendly  and  enemy  icons  have 
been  linked  and  reorganized,  suggesting  that  they  have  been  processed.  This 
oval  represents  the  comprehension  of  the  data  elements  that  were  perceived. 
The  oval  is  embedded  in  an  amorphous  shape.  This  shape  suggests  that  there 
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are  other  possible  ways  the  data  could  be  linked  and  reorganized  that  would 
lead  to  alternative  mental  representations  of  the  data.  The  specific  compre¬ 
hension  achieved  by  the  decision  maker  is  a  function  of  the  data  that  has 
propagated  through  the  model  and  the  contents  of  the  lens.  The  ©figure  near 
the  bottom  of  the  amorphous  shape  of  Oval  5  represents  one  possible 
organizational  structure  of  the  friendly  data.  Note  that  the  configuration  of 
friendly  icons  (e.g.  (§) )  within  the  oval  is  not  necessarily  correct. 
The  ©figure  may  actually  be  the  more  veridical  comprehension  with  respect 
to  the  environment  depicted  in  Oval  1 .  The  same  situation  may  be  true  with 
respect  to  the  configuration  of  the  enemy  icons  inside  and  outside  Oval  5. 

The  final  oval  (Oval  6)  represents  the  projection  or  prediction  of  the 
decision  maker.  This  prediction  is  based  on  what  has  been  comprehended  by 
the  decision  maker  (Oval  5)  and  in  the  way  the  decision  maker’s  lens  (Lens 
C)  affects  that  prediction.  Alternate  views  of  the  battlefield  (e.g.@)  within 
the  amorphous  shape  but  outside  of  Oval  5  do  not  contribute  to  or  influ 
ence  the  prediction  of  the  decision  maker.  As  an  example,  note  that  the 
comprehension  of  the  enemy  in  Oval  5  (e.g.  0)  )  has  been  modified  and  is 
shown  as  (§)  in  Oval  6.  Some  of  the  attributes  of  the  enemy  that  are  modified 
may  be  location,  capabilities,  and  intentions.  Other  comprehension  elements 
(e.g.  friendly  forces,  weather,  terrain,  etc.)  are  modified  according  to  their 
own  set  of  criteria.  Note  that  the  amorphous  shape  which  surrounds  Oval  6 
is  larger  than  that  which  surrounds  Oval  5  and  contains  even  more 
alternatives.  This  representation  depicts  the  idea  that  there  is  much  greater 
uncertainty  associated  with  prediction.  Thus,  what  is  actually  predicted  by 
the  decision  maker  is  only  one  possible  view  of  the  future.  The  further  into 
the  future  one  attempts  to  predict,  the  more  uncertain  will  be  the  prediction 
encapsulated  in  Oval  6. 

Situated  cognition  is  not  a  state  that  is  achieved  but  a  dynamic,  ongoing 
process  (Clancy  1997).  For  example,  the  sensor  coverage  and  communi¬ 
cations  network  are  dynamic.  The  sensor  coverage  is  shifting  constantly  as 
sensors  move  or  fly  about  the  environment,  resulting  in  changes  to  sensor 
coverage.  These  changes  in  coverage,  coupled  with  possible  communications 
outages,  will  affect  data  available  in  Ovals  2  and  3  and  ultimately  influence 
the  perception,  comprehension,  projection,  and  decision  making  of  the 
practitioner. 

For  the  sake  of  simplicity,  Figure  1  did  not  include  feedback  loops;  these 
feedback  loops  are  included  in  Figure  3  (above).  The  feedback  loops  represent 
not  only  the  iterative  and  dynamic  nature  of  the  model  but  also  attest  to  the 
decisions  made  by  practitioners.  These  feedback  loops  also  provide  insight 
into  the  cognitive  processing  and  the  decision  making  of  a  practitioner.  The 
feedback  loops  flow  from  Oval  5  (comprehension)  to  Ovals  1,2,  3,  and  4 
(environment,  sensors,  C2  workstation,  and  perception)  and  from  Oval  6 
(projection)  to  Ovals  1,  2,  3,  4,  and  5. 

The  following  example  illustrates  the  flow  through  the  model  and  the  role 
of  the  feedback  loops.  There  is  an  enemy  unit  moving  on  the  battlefield  (Oval 
1)  but  it  has  not  yet  been  detected.  At  some  point,  a  sensor  detects  the  motion 
of  the  enemy  unit  (Oval  2).  Data  from  the  sensor  are  sent  to  the  friendly  unit 
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Figure  3.  Feedback  Loops  in  the  Dynamic  Model  of  Situated  Cognition  represent  decisions  that  have  been  made 


and  appear  on  a  workstation  (Oval  3).  If  the  workstation  is  configured 
properly  and  the  decision  maker  is  attending  to  the  workstation  (based  on 
contents  of  Lens  A),  the  data  may  be  perceived  (Oval  4).  The  decision  maker 
may  comprehend  (Oval  5)  there  is  an  enemy  of  unknown  size  and  strength 
on  the  battlefield  based  on  the  local  context  and  experience  (as  well  as  other 
contents  of  the  Lens  B).  Given  that  comprehension  and  knowledge  of  enemy 
doctrine  (as  well  as  other  contents  of  Lens  C),  the  decision  maker  may  expect 
or  project  (Oval  6)  that  the  enemy  might  be  of  a  certain  size  and  moving  in 
a  particular  direction.  Having  made  that  projection,  the  decision  maker  may 
elect  to  reposition  an  unmanned  aerial  vehicle  (UAV)  to  more  closely  monitor 
the  enemy  unit.  This  decision  is  represented  by  the  feedback  loop  from  Oval 
6  to  Oval  2.  Once  the  UAV  arrives  on  station,  the  sensors  on  board  provide 
additional  data,  which  flow  from  Oval  2  to  Ovals  3, 4, 5,  and  6.  The  additional 
data  from  the  UAV  sensors  will  either  confirm  or  correct  the  earlier 
comprehension  and  projection. 

There  are  three  other  aspects  of  the  dynamic  model  of  situated  cognition 
that  have  not  been  fully  explained  or  illustrated  thus  far.  First,  there  are  other 
feedback  loops  in  the  model.  These  feedback  loops  extend  from  Oval  5 
(comprehension)  to  the  lenses  and  from  Oval  6  (projection)  to  the  lenses.  The 
rationale  for  these  feedback  loops  is  that  the  lenses  are  dynamic;  they  are 
constantly  changing.  The  comprehensions,  projections,  and  decisions  we 
make  contribute  to  the  manner  in  which  we  view  the  world.  For  example,  if 
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a  practitioner  believes  that  there  is  an  enemy  unit  on  the  battlefield  (Oval  5) 
then  the  lens  will  be  focused  (and  attention  will  be  directed)  to  look  for 
data  that  confirm  this  belief.  The  lens  causes  the  decision  maker  to  focus  on 
certain  data  elements  (while  excluding  other  data),  and  the  end  result  is  a 
change  in  the  perceptions,  comprehensions,  and  projections  experienced  by 
the  decision  maker. 

The  second  aspect  of  the  model  not  yet  discussed  is  the  role  played  by  other 
human  agents  in  the  complex  system.  We  believe  that  input  from  human 
agents  is  injected  into  the  model  between  Oval  3  and  Lens  A.  The  human 
agent  may  inject  a  perception  (‘Look  what  just  popped  up  on  my  radar 
screen’),  a  comprehension  (‘I  think  there  are  about  12  tanks  out  there.  It’s 
probably  an  enemy  company’),  or  a  projection  (‘Looks  like  they’re  planning 
to  attack  that  village’).  The  lens  of  the  decision  maker  influences  the  extent 
to  which  input  from  these  other  human  agents  are  considered.  If  the  decision 
maker  has  found  a  human  agent  to  be  particularly  unreliable,  then  the  lens 
of  that  decision  maker  may  lean  towards  filtering  out  or  discounting  that 
agent’s  input. 

The  third  aspect  not  fully  discussed  thus  far  is  the  nature  of  accuracy  and 
uncertainty  in  the  model.  Accuracy  and  uncertainty  often  interact  with  one 
another  throughout  the  model.  In  an  ideal  world,  data  are  transmitted  from 
one  oval  to  the  next  in  an  accurate  manner.  However,  as  suggested  earlier, 
data  can  be  missed  or  misidentified  so  that  they  are  no  longer  veridical  with 
respect  to  Oval  1 .  Accuracy  (or  lack  thereof)  is  a  characteristic  of  both  the 
technological  agents  and  human  agents  in  any  complex  system  and  is  there¬ 
fore  included  in  the  dynamic  model  of  situated  cognition. 

Lipshitz  and  Strauss  (1996:  189)  characterize  uncertainty  as  ‘a  sense  of 
doubt  that  blocks  or  delays  action’.  McCloskey  (1996)  and  Klein  (1999)  list 
four  sources  of  uncertainty:  missing  information;  unreliable  information; 
ambiguous  or  conflicting  information;  and  complex  information.  We  believe 
that  uncertainty  can  be  present  in  the  technological  portion  of  the  model  as 
well  as  the  perceptual  and  cognitive  portion  of  the  model.  For  example,  tech¬ 
nological  systems  may  ‘experience’  uncertainty  and  even  attempt  to  reduce 
uncertainty  if  data  are  missing,  unreliable,  or  ambiguous.  Present  technological 
systems  may  be  less  able  to  ‘cope’  with  uncertainty  (i.e.  are  more  brittle)  if 
the  uncertainty  is  due  to  conflicting  information  or  the  complexity  of  the 
information.  Humans,  on  the  other  hand,  have  the  potential  to  experience  and 
to  cope  with  all  four  sources  of  uncertainty. 

The  matrix  on  the  left  side  of  the  Figure  4  depicts  the  four  possibilities  that 
result  when  high  and  low  levels  of  accuracy  interact  with  high  and  low  levels 
of  certainty.  Amorphous  shapes  A-D  illustrate  four  possible  comprehensions 
or  projections  (Ovals  5  or  6)  and  the  interaction  between  accuracy  and 
uncertainty.  The  small  white  circles  in  each  shape  are  possible  projections 
whereas  the  black  circle  is  the  most  accurate  projection  (based  on  the  exact 
contents  of  Oval  1).  The  large,  off-center  oval  in  A  indicates  the  decision 
maker  is  neither  accurate  nor  certain.  In  B,  the  decision  maker  is  accurate  but 
still  uncertain.  In  C,  the  decision  maker  is  very  certain,  but  also  highly  inac¬ 
curate.  In  D,  the  decision  maker  is  both  accurate  and  certain.  This  interaction 
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Figure  4. 
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between  accuracy  and  uncertainty  occurs  throughout  the  model  (i.e.  Ovals 
1-6).  When  either  accuracy  or  certainty  are  low,  the  data  or  information  that 
flow  on  to  the  next  portion  of  the  model  will  have  debilitating  consequences 
on  all  that  follows  thereafter.  Having  described  the  dynamic  model  of  situated 
cognition,  the  next  task  is  to  consider  a  method  that  can  be  used  to  trace  the 
flow  of  activity  through  the  model. 


Applying  Process  Tracing  to  the  Assessment  of  Human-System 
Performance 

Technologists  and  human  factors  practitioners  tend  to  approach  the  assessment 
of  human-system  performance  from  different  perspectives.  Technologists 
often  want  to  compare  the  difference  between  the  ground  truth  or  data  in  the 
environment  (Oval  1)  with  what  the  sensor  array  has  detected  (Oval  2).  At 
best,  this  could  be  described  as  ‘technological  performance’  but  it  is  not 
human-system  performance.  Even  a  comparison  between  the  data  in  the 
environment  (Oval  1)  and  what  is  displayed  on  the  decision  maker’s  work¬ 
station  (Oval  3)  misses  the  mark.  It  is  tempting  for  technologists  to  make  these 
comparisons  because  (at  least  in  the  case  of  simulations)  the  contents  of  Ovals 
1 ,  2,  and  3  can  be  measured  with  precision.  However,  such  comparisons  do 
not  give  an  accurate  assessment  of  a  decision  maker’s  situated  cognition. 

In  recent  years,  human  factors  practitioners  have  developed  a  variety  of 
methods  to  assess  situated  cognition  and  situation  awareness  (see  Chipman 
et  al.  2000;  Gawron  2000).  These  methods  vary  in  their  degree  of  subjectivity, 
rigor,  and  intrusiveness.  Some  methods  attempt  to  compare  the  data  in  the 
environment  (Oval  1)  with  the  decision  maker’s  perception  (Oval  4),  compre¬ 
hension  (Oval  5),  and  projection  (Oval  6).  These  comparisons  can  result  in  a 
more  accurate  assessment  of  a  decision  maker’s  mental  model  of  the 
environment  than  the  methods  proposed  by  technologists.  However,  they  are 
still  problematic.  They  do  not  describe  how  the  decision  maker’s  under¬ 
standing  evolved.  They  do  not  account  for  the  technological  processes  (Ovals 
2  and  3)  that  contribute  to  situated  cognition.  Rather  than  adapting  either  a 
technological  or  a  perceptual/cognitive  approach  to  assessing  cognition,  what 
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is  needed  is  a  human-system  performance  (HSP)  approach.  Process  tracing 
is  one  method  that  provides  such  a  comprehensive  approach  to  assessing  HSP, 
as  well  as  an  approach  for  assessing  situated  cognition. 

Woods  (1993)  describes  process  tracing  as  follows. 

‘The  goal  in  these  methods  is  to  map  out  how  the  incident  unfolded  including  available 
cues,  those  cues  actually  noted  by  participants,  and  participants’  interpretation  in  both 
the  immediate  and  in  the  larger  institutional  and  professional  contexts.  This  is  called 
a  process  tracing  . . .  method  because  it  focuses  on  how  a  given  outcome  came  about.’ 

In  addition,  he  states,  ‘The  specific  techniques  within  this  family  are  all 
oriented  towards  externalizing  internal  processes  or  producing  external  signs 
that  support  inferences  about  internal  workings.’ 

Process  tracing  involves  using  multiple  data  collection  methods  throughout 
the  human-machine  system  (Ovals  1  through  6).  These  data  collection 
methods  should  be  as  continuous  as  possible.  Admittedly,  this  approach  is 
easier  to  implement  during  simulations  than  during  free-play  held  exercises 
or  actual  operational  settings.  However,  with  recent  improvements  in  data 
collection  techniques  and  monitoring  devices,  it  is  becoming  easier  to  collect 
continuous  data  even  in  operational  settings. 

Researchers  need  to  know  ground  truth  (Oval  1).  While  in  simulations, 
ground  truth  can  be  known  in  real  time.  However,  in  operational  settings 
ground  truth  can  be  determined  only  after  the  fact  and  this  determination  may 
be  imprecise.  Researchers  also  need  access  to  sensor  data  (Oval  2).  These 
data  include  where  the  sensors  are  located,  their  operating  characteristics  (e.g. 
sensor  type,  range,  etc.),  and  the  data  they  are  collecting.  The  data  detected 
by  the  sensor  array  place  an  upper  limit  on  human  cognition.  A  decision 
maker’s  knowledge  of  the  environment  can  be  no  better  than  what  has  been 
detected  by  the  sensors.  Investigators  should  know  how  the  decision  makers’ 
local  workstations  are  configured  (Oval  3).  Further,  they  need  to  understand 
why  the  decision  makers  configured  the  workstations  in  that  way  and  know 
what  data  are  (and  are  not)  available  to  the  decision  makers  because  of  those 
configurations.  Again,  Oval  3  represents  a  further  constraint  on  human 
cognition.  At  any  given  point  in  time  understanding  can  be  no  better  than  the 
data  the  decision  makers  are  able  to  access. 

Data  collection  in  the  technological  portion  of  the  system  (Ovals  1  through 
3)  involves  closely  monitoring  activities  as  they  unfold  and  harvesting  data 
from  the  system  after  the  fact.  Researchers  can  greatly  improve  the  quality 
of  data  they  will  harvest  if  they  work  with  designers  and  engineers  before  the 
data  collection  event  to  ensure  that  the  appropriate  activities  are  recorded  and 
in  the  correct  format  and  that  timing  devices  are  synchronized.  Data  collection 
during  the  perceptual  and  cognitive  portion  of  the  system  (Ovals  4  through 
6)  involves  monitoring  and  measuring  the  behaviors  of  the  decision  makers 
with  as  little  intrusion  as  possible. 

Researchers  must  determine  the  contents  of  each  decision  maker’s  lens. 
While  it  may  be  impossible  to  correctly  assess  the  entire  contents  of  each 
individual’s  lens,  the  contents  can  be  assessed  with  tests  and  interviews  before 
and  after  the  activity  being  investigated.  Eye-tracking  technology  with  gaze 
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analysis  can  be  used  to  help  determine  where  decision  makers  were  focusing 
their  attention.  Physiological  monitoring  can  identify  changes  in  physiological 
stress  levels  of  the  decision  makers.  Voice  and  text  transcripts  can  be  used  to 
determine  how  data  were  integrated  into  comprehension  and  how  comprehen¬ 
sion  led  to  projections  about  the  future.  Activities  such  as  asking  decision 
makers  to  briefly  sketch  the  most  important  aspects  of  the  environment  they 
are  monitoring  can  be  done  quickly  with  minimal  disruption  but  will  provide 
rich  insight  into  the  focus  of  attention  and  other  thought  processes  of  the 
decision  makers. 

These  data  collection  methods,  if  used  to  trace  the  flow  of  data  through  the 
technological  and  human  portions  of  a  system,  will  not  only  describe  how  the 
decision  makers  came  to  understand  what  was  happening  in  their  environment 
but  whether  and  how  that  understanding  went  awry.  These  methods  are 
consistent  with  Woods’  view  of  process  tracing:  ‘Process-tracing  techniques 
primarily  use  data  from  verbal  reports  or  from  records  of  problem-solver 
behavior  to  build  protocols  that  describe  the  sequence  of  information  flow 
and  knowledge  activation’  (Woods  1993).  Results  of  this  process-tracing 
method  are  also  useful  for  informing  designers  where  problems  exist  in  the 
technological  portion  of  the  system. 

The  next  section  illustrates  the  utility  of  the  dynamic  model  of  situated 
cognition  by  applying  it  to  the  collision  between  the  submarine  USS 
Greeneville  and  the  Japanese  motor  vessel  Ehime  Maru  on  February  9, 2001 . 
This  retrospective  analysis  traces  the  flow  of  data  through  the  technological 
and  human  components  of  the  submarine’s  crew.  This  case  study  not  only 
describes  what  the  crewmembers  knew  but  also  how  the  technology  prevented 
them  from  knowing  critical  pieces  of  information.  The  authors  then  contrast 
the  analysis  of  this  accident  using  the  DMSC  with  the  analysis  that  other  NDM 
theories  might  yield.  It  is  our  contention  that  examining  decision  making  with 
a  model  that  includes  both  technological  and  human  components  yields  a  more 
robust  and  more  accurate  explanation  of  the  events. 


USS  Greeneville  (SSN  772) 

On  February  9, 2001.  the  US  Navy  submarine  USS  Greeneville  collided  with 
the  Japanese  motor  vessel  Ehime  Maru  off  the  coast  of  Oahu,  Hawaii.  The 
submarine  was  demonstrating  an  emergency  surfacing  maneuver  for  civilian 
guests  onboard  for  a  seven-hour  distinguished  visitor  cruise.  As  it  rose  to  the 
surface,  the  submarine  struck  the  fishing  vessel’s  aft  port  quarter,  causing 
the  ship  to  sink  in  less  than  10  minutes.  Of  the  35  Japanese  crew,  instructors, 
and  students  onboard  the  Ehime  Maru,  26  were  rescued  while  nine  remain 
unaccounted  for,  presumed  dead. 

The  problems  started  when  the  submarine  had  fallen  behind  schedule  by  30 
minutes  and  had  less  than  an  hour  to  get  to  a  pre-designated  location.  There 
were  three  surface  ships  in  the  vicinity  of  the  submarine.  The  Ehime  Maru  was 
closing  on  the  USS  Greeneville  but  at  this  point  this  information  only  resided 
in  Oval  1  (ground  truth  or  data  in  the  environment)  and  in  Oval  2  (data  detected 
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by  sensors)  of  the  dynamic  model  of  situated  cognition.  The  information  on 
the  closing  rate  of  the  Ehime  Maru  was  not  in  Oval  3  (data  available  on  the 
submarine’s  C2  system)  because  a  critical  display  system  was  not  working. 
The  analog  video  signal  display  unit,  located  in  the  control  room  of  the 
submarine,  provides  a  remote  display  of  sonar  data  used  for  surface  contact 
analysis.  It  was  inoperative.  Since  information  on  the  proximity  of  the  Ehime 
Maru  was  not  available  in  Oval  3  from  the  analog  video  signal  display  unit,  it 
could  not  be  propagated  throughout  the  rest  of  the  model.  At  this  point,  the 
commanding  officer  of  the  USS  Greeneville  made  a  series  of  decisions  and 
issued  orders  that  created  an  artificial  sense  of  urgency  in  crewmembers  in  the 
control  room.  This  elevated  time  pressure  affected  the  individual  lenses  (Lens 
A,  B,  and  C  on  the  right  side  of  the  model)  of  the  crewmembers,  adversely 
impacting  their  ability  to  accurately  process  the  information  residing  in  Oval 
4  through  Oval  6  (perception,  comprehension,  and  projection). 

One  of  the  decisions  made  by  the  commanding  officer  was  to  prepare  to 
come  to  periscope  depth.  Mandatory  procedures  for  a  submarine  to  come  to 
periscope  depth  require  that  the  officer  of  the  deck  hold  a  periscope  briefing 
with  watchstanders,  conduct  two  good  target  motion  analysis  legs  of  about 
three  minutes  on  each  surface  contact,  provide  the  necessary  report,  and 
obtain  the  commanding  officer’s  permission  to  proceed  to  periscope  depth. 
These  procedures  were  known  to  all  crewmen  (i.e.  were  part  of  their  lenses). 
The  commanding  officer,  however,  abbreviated  these  procedures  used  by  the 
crew  to  maintain  their  situation  awareness  during  the  periscope  depth 
maneuver.  The  commanding  officer’s  decision  not  only  compromised  the 
procedures,  it  virtually  assured  that  the  data  in  the  environment  (Oval  1) 
detected  by  the  sensors  (Oval  2)  would  be  inaccurate  or  incomplete.  Hence, 
the  data  displayed  on  the  C2  workstations  (Oval  3)  would  also  be  inconsistent 
with  Oval  1;  and  the  perception,  comprehension,  and  projection  (Ovals  4,  5, 
and  6)  of  the  crewmembers  with  regard  to  the  surface  contacts  would  be 
formulated  based  on  erroneous  data. 

Prior  to  the  commanding  officer’s  decision  to  surface,  the  sonar  technician 
reported  a  new  contact  to  the  control  team.  This  information  from  the 
technician  either  did  not  pass  through  Lens  A  of  the  other  human  agents  or 
was  skewed  as  it  passed  through.  The  result  was  that  neither  the  commanding 
officer  nor  the  officer  of  the  deck  perceived  (Oval  4)  the  situation  properly; 
they  did  not  recognize  that  the  sonar  report  was  information  on  a  new  contact. 
The  commanding  officer  then  announced  to  the  control  room  that  he  had  a 
good  feel  for  the  contact  picture  and  ordered  the  officer  of  the  deck  to  proceed 
to  periscope  depth  on  the  same  course.  The  officer  of  the  deck  was  not  given 
enough  time  to  develop  an  accurate  picture  of  the  surface  contact  situation. 
He  did  not  conduct  the  required  periscope  brief  with  watchstanders,  missing 
a  valuable  opportunity  to  receive  and  critically  assess  important  contact 
information  from  the  sonar.  He  was  deprived  of  input  from  both  the  tech¬ 
nological  agents  and  the  human  agents  in  the  system. 

Additionally,  other  crewmembers  were  not  given  enough  time  to  do  their 
jobs  properly.  Upon  hearing  the  sonar  technician’ s  report  of  the  new  contact, 
the  fire  control  technician  of  the  watch  rushed  to  complete  his  analysis  of  three 
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surface  contacts  prior  to  periscope  depth,  overlooking  an  updated  4000-yard 
closing  solution  on  one  of  the  old  contacts  (the  ill-fated  Ehime  Maru).  His 
focus  was  entirely  on  a  ’new‘  contact  which  he  considered  to  be  the  primary 
contact  of  interest.  The  fire  control  technician  of  the  watch’s  lens  was  skewed 
by  the  false  sense  of  urgency  established  by  the  commanding  officer  which 
could  very  well  have  narrowed  the  fire  control  technician  of  the  watch’ s  focus 
of  attention.  Further,  when  the  fire  control  technician  of  the  watch  heard  the 
commanding  officer  say  he  had  a  good  feel  for  the  contact  picture,  he  assumed 
the  commanding  officer  was  referring  to  all  contacts,  including  the  new  one. 
This  provides  further  evidence  of  a  skewed  lens  on  the  part  of  the  fire  control 
technician  of  the  watch,  which  may  have  contributed  to  his  decision  to  remain 
silent,  failing  to  provide  the  commanding  officer  with  corrective  information. 

The  commanding  officer’s  erroneous  perception  led  to  an  incorrect  compre¬ 
hension  (Oval  5)  of  the  situation  and  an  inaccurate  projection  (Oval  6)  of  where 
the  vessels  would  be  in  the  future.  At  no  time  did  the  commanding  officer 
discuss  the  surface  picture  with  the  contact  management  team  to  verify  a 
common  understanding  of  the  surface  contacts  (Ovals  5  and  6,  comprehension 
and  projection).  His  own  situation  awareness  of  contacts  was  based  on  two 
brief  walkthroughs  of  the  sonar  room  and  a  single  review  of  fire  control 
displays.  He  was  overconfident  and  pressed  for  time,  and  failed  to  properly  use 
both  the  technological  agents  and  the  human  agents  in  the  system  to  build  his 
understanding  of  the  surface  picture.  The  decision  to  proceed  to  periscope  depth 
represents  a  feedback  loop  from  Oval  6  (projection)  to  Oval  1  (environment). 
As  the  submarine  ascended  to  periscope  depth,  the  contents  of  Oval  1  were 
changed  and  new  data  were  available  for  propagation  through  the  model. 

While  at  periscope  depth,  the  commanding  officer  decided  to  interrupt  the 
officer  of  the  deck’s  periscope  search  and  performed  his  own  abbreviated  visual 
search  for  surface  contacts.  After  the  periscope  searches  by  the  officer  of  the 
deck  and  commanding  officer,  the  fire  control  technician  of  the  watch  cycled 
back  through  his  surface  contacts  and  correctly  calculated  a  dangerous  closing 
solution  for  one  of  the  contacts,  the  Ehime  Maru.  However,  the  officer  of  the 
deck  had  just  stated  he  had  seen  no  close  contacts  at  periscope  depth,  and  the 
commanding  officer  also  said  he  had  no  visual  contacts.  These  pronouncements 
so  skewed  the  lens  of  the  fire  control  technician  of  the  watch  that  he  doubted 
his  comprehension  (Oval  5)  of  the  situation  (that  there  was  a  surface  contact 
in  close  proximity  to  the  submarine).  The  fire  control  technician  of  the  watch’s 
erroneous  comprehension  generated  a  decision  and  a  feedback  loop  from  Oval 
5  (comprehension)  to  Oval  3  (C2  workstation).  He  manually  overrode  the 
correct  solution  presented  by  his  workstation,  physically  changing  the  distance 
of  the  surface  contact  ( Ehime  Maru )  from  4000  to  9000  yards,  reflecting  the 
distance  to  the  visual  horizon.  The  fire  control  technician  of  the  watch  entered 
this  erroneous  information  into  the  fire  control  system.  The  result  of  this  action 
was  that  the  speed  solution  of  the  surface  contact  resulted  in  an  impossible 
speed  solution  of  99  knots.  After  the  periscope  searches,  the  boat  went  ‘emer¬ 
gency  deep’ ,  proceeded  to  400  feet,  and  conducted  an  emergency  main  ballast 
tank  blow.  The  ship  surfaced  underneath  the  Ehime  Maru,  causing  major 
flooding  on  that  ship  which  sunk  rapidly. 
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Application  of  NDM  to  the  USS  Greeneville  Accident 

At  the  beginning  of  this  paper,  we  described  the  conditions  in  which  NDM 
is  likely  to  be  employed.  These  conditions  include  ill-structured  problems, 
uncertain  dynamic  environments,  ill-defined  or  shifting  goals,  action  or 
feedback  loops,  time  stress,  high  stakes,  multiple  players,  and  organizational 
goals  and  norms  (Orasanu  and  Connolly  1993).  All  of  these  conditions  were 
present  on  the  USS  Greeneville.  Given  the  presence  of  these  conditions,  it  is 
appropriate  to  consider  how  our  extension  of  NDM,  which  addresses 
cognition  of  multiple  agents  in  a  complex  system,  compares  with  present 
NDM  models. 

We  submit  that  the  dynamic  model  of  situated  cognition  has  advantages 
over  current  NDM  models  in  at  least  four  areas. 

1  Technological  aspects  of  complex  systems.  When  the  dynamic  model  of 
situated  cognition  is  used  to  describe  the  events  which  took  place  on  the 
USS  Greeneville,  it  is  clear  that  the  accident  is  a  result  not  only  of 
decisions  made  by  human  agents  but  also  of  the  data  that  flowed  through 
the  technological  agents  to  the  human  agents  in  the  complex  system. 
Current  NDM  models  discuss  the  ‘environment’  but  none  really 
acknowledge  the  tight  coupling  of  the  technological  and  human  agents 
in  complex  systems.  In  the  USS  Greeneville  accident,  NDM  models 
might  begin  the  analysis  by  considering  what  data  were  available  to  the 
human  rather  than  what  data  resided  in  the  system. 

2  Data  flow.  The  dynamic  model  of  situated  cognition  depicts  the  flow  of 
data  from  the  external  environment  through  the  technological  agents  to 
the  human  agents.  The  model  makes  it  apparent  where  data  have  been 
blocked,  missed,  or  altered  as  they  propagate  through  the  system.  NDM 
models  are  excellent  for  describing  how  a  practitioner  arrived  at  a  deci¬ 
sion  in  complex  system  based  on  the  data  available.  In  the  USS 
Greeneville,  NDM  models  would  typically  start  with  the  data  that  were 
available  on  the  crew  workstations  (Oval  3).  However,  whether  designing 
a  complex  system  or  investigating  a  mishap  in  a  complex  system,  it  is 
necessary  to  be  cognizant  of  what  could  have  been  known  and  what  should 
have  been  known  by  both  the  technological  and  human  agents  throughout 
the  entire  system. 

3  Multiple  players.  Although  the  conditions  in  which  NDM  is  likely  to  be 
employed  normally  include  multiple  players,  many  NDM  models  describe 
the  activities  of  a  single  decision  maker.  In  general,  NDM  models  are  able 
to  describe  the  decisions  made  by  the  commanding  officer  or  the  officer 
of  the  deck  but  are  less  able  to  describe  the  interactions  among  multiple 
practitioners  and  the  decisions  they  make.  The  dynamic  model  of  situated 
cognition  takes  into  account  the  fact  that  there  are  multiple  human  and 
machine  agents  in  a  complex  system  and  that  these  agents  interact  with 
one  another  in  ways  which  affect  the  decision-making  process. 

4  Dynamic  nature  of  decision  makers.  NDM  models  consider  the  role  of 
expertise.  In  Klein’s  RPD  model,  greater  expertise  results  in  the  decision 
maker  having  experienced  a  greater  number  of  similar  instances  with 
which  to  compare  the  current  situation.  In  general,  however,  current  NDM 
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models  do  not  adequately  address  individual  states  and  traits  or  social 
factors.  Aboard  the  USS  Greeneville,  although  the  commanding  officer 
was  an  experienced  decision  maker,  he  was  probably  stressed  from  being 
behind  schedule  and  from  his  desire  to  impress  the  distinguished  visitors. 
He  communicated  a  sense  of  urgency  to  his  crew  with  his  decisions  and 
his  actions.  The  dynamic  model  of  situated  cognition  takes  such  factors 
into  account.  The  contents  of  the  lenses  include  experience,  social  factors, 
individual  states  and  traits,  the  local  context,  specific  plans,  and  broad 
guidelines.  These  elements  permit  analysis  of  decisions  based  on  both 
transient  individual  and  situational  characteristics  as  well  as  enduring 
individual  and  organizational  characteristics. 

Conclusion 

We  believe  the  dynamic  model  of  situated  cognition  extends  current  NDM 
models  in  a  significant  manner  —  by  fully  considering  the  technological 
portion  of  a  complex  system.  The  model  we  propose  offers  an  innovative  way 
to  view  complex  systems  in  which  humans  and  machines  function  as  coopera¬ 
tive  agents.  The  dynamic  model  of  situated  cognition  recognizes  the  unique 
contributions  made  by  both  the  technology  and  humans.  By  employing 
process  tracing  methods  to  track  the  flow  of  data  through  the  model,  it  is  easy 
to  determine  where  the  data  may  have  been  blocked,  how  lenses  may  have 
been  skewed,  and  how  human  agents  arrived  at  erroneous  perceptions, 
comprehensions,  and  projections,  which  then  resulted  in  poor  decisions. 

We  have  used  this  model  previously  to  describe  events  in  numerous  com¬ 
plex  systems.  However,  the  model  might  also  be  used  in  a  predictive  manner. 
For  example,  the  model  could  be  used  to  guide  system  development. 
Designers  who  want  to  maximize  system  performance  need  to  consider  both 
sides  of  the  model.  They  need  to  ensure  sufficient  sensors  are  available  to 
detect  data  in  the  environment.  Data  from  the  sensors  need  to  be  relayed  to 
the  C2  workstations  and  displayed  on  the  workstations  in  a  manner  that  will 
not  overload  decision  makers.  And  decision  makers  need  to  be  aided  so  that 
their  perceptions,  comprehensions,  projections,  and  decisions  facilitate  the 
accomplishment  of  the  system’s  goals. 
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